Devices Having Calibrated LEDs

ABSTRACT

The various implementations described herein include methods, devices, and systems for calibrating LED(s). In one aspect, an electronic device includes: a plurality of light emitting diodes (LEDs); one or more processors; and memory storing a plurality of color correction matrices, each color correction matrix of the plurality of color correction matrices corresponding to an LED of the plurality of LEDs and generated based on a desired color value for the corresponding LED, wherein the electronic device is configured to relay status of the electronic device via the plurality of LEDs operating in conjunction with the plurality of color correction matrices.

PRIORITY

This application is a continuation of U.S. patent application Ser. No.16/141,778, filed Sep. 25, 2018, which is a continuation of U.S. patentapplication Ser. No. 15/717,751, filed Sep. 27, 2017, now U.S. Pat. No.10,111,296, issued Oct. 23, 2018, which claims priority to U.S.Provisional Patent Application No. 62/403,639, filed Oct. 3, 2016, eachof which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

This application relates generally to illumination devices, includingbut not limited to methods and systems for calibrating color lightemitting diodes (LEDs).

BACKGROUND

LEDs, if not calibrated, come with large amount of variation inbrightness and color. To achieve better uniformity in brightness andcolor, LED manufacturers often employ a process called “binning.” In thebinning process, LEDs of similar brightness and color are “binned”together. The LED manufactures often put LEDs of the same bin into thesame reel to facilitate manufacturing requirements of brightness andcolor uniformity. The variance within the same bin of LEDs depends on anumber of parameters such as price, available production volume, andetcetera. Usually, at reasonable cost, each bin is binned to an accuracyof about 15% for green (the color to which the human eye is mostsensitive) and 50-100% for blue (the colors to which the human eye isless sensitive), and a buyer is usually required to purchase multiplebins. However, even if binned LEDs are used, the resulting worst casecolor variation is still unacceptable when distinctive or specificcolors are required.

SUMMARY

Accordingly, there is a need for systems and/or devices with moreefficient, precise, accurate, and cost-effective methods for configuringLEDs for consistent color and/or brightness. Such systems, devices, andmethods optionally complement or replace conventional systems, devices,and methods for configuring LEDs.

The conventional method of binning is a brute-force methodology; it canbe effective but is also costly. The cost lies in the overhead ofselecting the LEDs and the resulting more complex inventory management.The disclosed implementations include a LED calibrating methodology. Thedisclosed implementations do not rely on brute-force techniques such asbinning but instead analyze variations in color and/or brightness ofindividual LEDs (including LEDs with wide variations in brightnessand/or color) and then calculate correction factors needed to eliminatethe variations altogether for those individual LEDs. The correctionfactors can be saved and then used to program/configure LED drivers toprovide consistent color and/or brightness of light output fromrespective individual LEDs during use. As a result, in applicationswhere consistency of LEDs is important (e.g., in an electronic deviceincluding a group of LEDs that are required to provide illumination withspecific color component values), the disclosed implementations canprovide such consistency using a combination of LEDs with inherent colorand/or brightness variations by applying the individual correctionfactors during use to correct for those individual variations.

Advantages of solving the problem using the disclosed implementationsinclude improving the quality consistency of LEDs and cost savings thatresult from correcting inherent variations in LEDs withoutlabor-intensive processes, such as binning. The disclosedimplementations improve LED color and brightness uniformity. Thedisclosed implementations also allow for the use of cheaper LEDcomponents to achieve the illumination uniformity associated with moreexpensive LED. All products with LED lighting can benefit from thedisclosed calibrating methodology, including LED displays, OLED displaysand the like.

The various implementations described herein include methods, devices,and systems for calibrating LED(s). In one aspect, a method includes:(1) obtaining a desired color value for each LED of a plurality of LEDsto be calibrated; (2) obtaining image information from an image sensor,the image information corresponding to operation of the plurality ofLEDs; and (3) generating calibration information for each LED of theplurality of LEDs based on the desired color value for the LED and theobtained image information.

Thus, devices and systems are provided with methods for configuringLEDs, thereby increasing the effectiveness, efficiency, accuracy,precision, and user satisfaction with such systems. Such methods maycomplement or replace conventional methods for configuring LEDs.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

For a better understanding of the various described implementations,reference should be made to the Description of Implementations below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIG. 1A shows diagram of a representative LED calibration system inaccordance with some implementations.

FIGS. 1B and 1C show prophetic LED colors before and after calibrationrespectively, in accordance with some implementations.

FIG. 1D is a flow diagram illustrating a method of calibrating an LEDdevice in accordance with some implementations.

FIGS. 2A and 2B are a front view and a rear view of a voice-activatedelectronic device in accordance with some implementations.

FIG. 2C is a top view of a voice-activated electronic device inaccordance with some implementations.

FIG. 2D shows six visual patterns displayed by an array of full colorLEDs for indicating voice processing states in accordance with someimplementations.

FIG. 3 is a block diagram illustrating an example electronic deviceutilizing LEDs in accordance with some implementations.

FIG. 4 is a flow diagram illustrating a method of visually indicating avoice processing state in accordance with some implementations.

FIG. 5 is a block diagram illustrating an example calibrating device inaccordance with some implementations.

Like reference numerals refer to corresponding parts throughout theseveral views of the drawings.

DESCRIPTION OF IMPLEMENTATIONS

Light emitting diodes (LEDs) come with large variations in color andbrightness (e.g., variations noticeable to the human eye). Someelectronic devices include components that use LEDs to provideillumination or to display information. For example, in the Google Homevoice- activated electronic device the LED components include 12 fullcolor LEDs that are selectively illuminated to show different kinds ofanimations during user interactions that are used to indicate the voiceprocessing state of the Google Home device and the back end userassistant for which it is the front end, and of the user interactions.In some operating conditions, these LED illumination components areexpected to display Google brand colors and thus it is important from abranding point of view to keep the color consistent across all units andwithin individual pixels of particular unites. However, given the highlevel of variations in color and brightness of un-binned LEDs (e.g.,some LEDs will appear more red than others, some LEDs will appear moreblue than others etc.) may affect consumers' perception of the Googlebrand colors or the quality of the Google Home product. For example, ifthe Google brand colors appear too inconsistent across units, then thebrand message may lose its clarity.

In accordance with some implementations, an electronic device includesan array of full color light emitting diodes, each of which comprisesthree individually controllable LEDs, each of which providesillumination for corresponding color space components, such as Red,Green and Blue color components. As is well known, the color perceivedby a user resulting from illumination of an individual full color LED isdetermined largely by the combination of the individual color values ofthe three component LEDs. While the electronic device processes inputs,the array of full LEDs is illuminated to provide a visual patternaccording to LED illumination specifications determined according to astate of the processing. The array of full color LEDs is configured toprovide a plurality of visual patterns each corresponding to aprocessing state. The LED design language used to create the visualpatterns is applied to at least partially resolve the problem of userconfusion, apprehension, and uneasiness and promote understanding,adoption and enjoyment of the corresponding interface experience.Therefore it is important to ensure that the LEDs have consistent colorand brightness lest important information being conveyed by the LEDindicators on the device be misrepresented, or misconstrued by a user.

FIG. 1A shows a diagram of an LED calibration system 100 in accordancewith some implementations. In some implementations, the calibrationsystem is 100 utilized to calibrate the LEDs of one or more LED devices.In some implementations, the calibration system calibrates (alsosometimes called tuning or configuring) the LEDs individually on eachLED device. In some implementations, the LEDs are calibrated prior tobeing installed in a device—e.g., after being mounted on a PCB but priorto that PCB being mounted in a device during a manufacturing operation.In some implementations, the tuning system includes: (1) a calibrationcontainer 102 with a dark interior configured to block all outside lightfrom entering the container; (2) an image sensor 104 (e.g., camera)mounted inside the container 102; and (3) a calibrating device 106(e.g., a laptop) coupled to the image sensor 104 and the LED device 190being calibrated and configured to determine calibration parameters forthe LED device 190. In some implementations, the calibration system 100is configured to calibrate a plurality of LED devices 190 concurrently.In some implementations, the LED device consists of one or more LEDs andstorage for storing the calibration parameters. In some implementations,the LED device comprises an LED module configured.

In some implementations, the calibrating device 106 includes memorystoring one or more programs, which contain instructions to extract LEDvariation data and calculate the calibration parameters. In someimplementations, reading pictures and calculating calibration parametersis automated.

In some implementations, the LED device (e.g., device 190) includesmemory for storing the calibration data (e.g., LED configuration data336). In some implementations, the LED device includes an LED controller(e.g., LED control module 324) configured to utilize the calibrationdata to make the LEDs uniform in both color and brightness duringillumination of the LEDs (either as a group or individually).

In some implementations, inside the calibration container 102, the imagesensor 104 takes a series of pictures of the LEDs on the LED devicebeing calibrated. The resulting images are then analyzed by thecalibrating device 106. The calibrating device 106 analyzes each LED'scolor and brightness and sends correction values to the LED device foreach LED. The result is that all the LEDs have more consistent color andbrightness (e.g., less than a particular threshold in variation). Insome implementations, the calibrating device 106 calibrates the LEDssuch that the color and brightness of each LED is within a particularthreshold of the desired values (e.g., within 2%, 5%, or 10% of aspecific color value). For implementations where the LEDs are fullcolor, the image sensor evaluates component values of each LED. Forexample, when the electronic device includes a ring of 12 LEDs, theimage sensor evaluates the individual Red, Green and Blue componentvalues of each of the 12 LEDs.

In some implementations, the image sensor 104 comprises a camera. Insome implementations, picture acquisition from the camera is completelyautomated. In some implementations, transferring the pictures from thecamera to the calibrating device 106 is automatic, while in otherimplementations, transferring the pictures to the calibrating device 106is manual. In some implementations, the image sensor 104 comprises acolor meter coupled with optical fiber. The data for the individual LEDcolor/brightness variation is collected by using the optical fiber andcolor meter.

FIGS. 1B and 1C show prophetic LED colors before and after calibrationrespectively, in accordance with some implementations. In FIG. 1B thetwo “red” LEDs are noticeably different from one another, likewise withthe “blue” LEDs, “green” LEDs, and “yellow” LEDs. In FIG. 1C the “red,”“blue,” “green,” and “yellow” LEDs are uniform in color as a result ofthe disclosed calibrating methodologies.

FIG. 1D is a flow diagram illustrating a method 120 of calibrating anLED device in accordance with some implementations. In someimplementations, the method 120 is performed by a calibrating device 106(FIG. 5) or a component thereof, such as a LED calibration module 518.In some implementations, the operations of the method 120 describedherein are entirely interchangeable, and respective operations of themethod 120 are performed by any of the aforementioned devices, systems,or combination of devices and/or systems. In some embodiments, method120 is governed by instructions that are stored in a non-transitorycomputer-readable storage medium and that are executed by one or moreprocessors of a device/computing system, such as the one or more CPU(s)502 of calibrating device 106. For convenience, method 120 will bedescribed below as being performed by the calibrating device 106.

In some implementations, the calibrating device 106 obtains (122)desired color values for each LED to be calibrated. For example, thecalibrating device 106 obtains the desired color values from a colordesigner or from a product specification.

In some implementations, the calibrating device 106 converts (124) thedesired color value to a color space (e.g., to an sRGB color spacewithout gamma encoding). In some instances, the desired color valuescomprise non-linear RGB values. In some implementations, the non-linearRGB values are converted to linear RGB values utilizing equation 1below.

$\begin{matrix}{{{De}\text{-}{Gamma}\mspace{14mu} {of}\mspace{14mu} {RGB}\mspace{14mu} {values}}{{\langle\begin{matrix}R_{sRGB} \\G_{sRGB} \\B_{sRGB}\end{matrix}\rangle}\overset{{De}\text{-}{Gamma}}{\Rightarrow}{\langle\begin{matrix}R_{{sRGB},{linear}} \\G_{{sRGB},{linear}} \\B_{{sRGB},{linear}}\end{matrix}\rangle}}} & {{Equation}\mspace{14mu} 1}\end{matrix}$

In the above equation, the sRGB subscript refers to RGB values specifiedin sRGB color space with gamma encoding. The sRGB,linear subscriptrefers to RGB values specified in sRGB color space without the gammaencoding. In some implementations, the de-gamma process utilizes alook-up table (e.g., a 256-value look-up table).

The calibrating device 106 obtains (126) image information (e.g., apicture) from the image sensor 104 corresponding to operation of theLED(s) within the calibration container. In some implementations, thecalibrating device 106 obtains a picture of the operating LEDs from theimage sensor 104. In some implementations, the image informationcorresponds to operation of the LEDs within a housing of the LED device.Therefore the calibration of the LEDs includes calibrating for potentialvariances caused by the LEDs' light passing through the housing. In someimplementations, the calibrating device 106 obtains image informationfor each color component of the full color LED's. For example, thecalibrating device 106 obtains a first picture capturing the redcomponent of the LED (e.g., a picture taken when only the red componentis active), obtains a second picture capturing the blue component of theLED, and obtains a third picture capturing the green component of theLED. In some implementations, the calibrating device 106 extracts theimage information of each color component from a single picture (e.g., apicture taken when all color components are active).

The calibrating device 106 obtains (128), from the image information, acolor matrix for each LED to be calibrated. In some implementations, thecolor matrix is obtained in the color space of the image sensor 104. Forexample, the image sensor 104 comprises a 1931 XYZ camera and the colormatrix is in the XYZ color space. In some implementations, obtaining theimage information from the image sensor includes obtaining one or morepictures from the image sensor. In some implementations, the calibratingdevice selects an optimal picture from the one or more obtainedpictures.

The calibrating device 106 generates (130), for each LED, a colorcorrection matrix (CCM) based on the color matrix for the LED. In someimplementations, generating a color correction matrix includesgenerating an offset matrix in the color space of the image sensor 104.In some implementations, generating a color correction matrix includesutilizing equation 2 below.

$\begin{matrix}{{{Color}\mspace{14mu} {Correction}\mspace{14mu} {Matrix}}{{{{INVERT}\left( {\langle\begin{matrix}X_{{RLED}^{\prime}} & X_{{GLED}^{\prime}} & X_{{BLED}^{\prime}} \\Y_{{RLED}^{\prime}} & Y_{{GLED}^{\prime}} & Y_{{BLED}^{\prime}} \\Z_{{RLED}^{\prime}} & Z_{{GLED}^{\prime}} & Z_{{BLED}^{\prime}}\end{matrix}\rangle} \right)} \times {\langle\begin{matrix}X_{{RsRGB}^{\prime}} & X_{{GsRGB}^{\prime}} & X_{{BsRGB}^{\prime}} \\Y_{{RsRGB}^{\prime}} & Y_{{GsRGB}^{\prime}} & Y_{{BsRGB}^{\prime}} \\Z_{{RsRGB}^{\prime}} & Z_{{GsRGB}^{\prime}} & Z_{{BsRGB}^{\prime}}\end{matrix}\rangle}} = {\langle\begin{matrix}C_{11} & C_{12} & C_{13} \\C_{21} & C_{22} & C_{23} \\C_{31} & C_{32} & C_{33}\end{matrix}\rangle}}} & {{Equation}\mspace{14mu} 2}\end{matrix}$

In the above equation, INVERT( )refers to the operation of inverting amatrix. [X, Y, Z] refers to the particular color components specified inthe color space; LED' refers to the color space of the image sensor; andsRGB' refers to the sRGB color space.

In some implementations, the calibrating device scales the CCM toprevent overrange (also sometimes called clipping) of the RGB values forthe LED. Scaling the CCM ensures that the resulting RGB values arewithin the operating range of the LED.

The calibrating device 106 sends (132) the color correction matrix foreach LED to the LED device. In some implementations, the calibratingdevice 106 sends the CCM to the LED device via a USB port on the LEDdevice. In some implementations, the calibrating device 106 sends one ormore brightness parameters for each LED to the LED device.

In some implementations, the LED device stores the CCM for futureoperation of the LEDs (e.g., LED configuration data 336). In someimplementations, the LED device stores the one or more brightnessparameters for each LED for future operation of the LEDs (e.g., LEDconfiguration data 336). In some implementations, the LED deviceutilizes the CCM to adjust the operation of the individual LEDs. Forexample, in accordance with some implementations, the LED deviceutilizes equation 3, below, to adjust the LEDs.

$\begin{matrix}{{{Adjusting}\mspace{14mu} {LED}\mspace{14mu} {output}}{{{\langle\begin{matrix}C_{11} & C_{12} & C_{13} \\C_{21} & C_{22} & C_{23} \\C_{31} & C_{32} & C_{33}\end{matrix}\rangle} \times {\langle\begin{matrix}R_{{sRGB},{linear}} \\G_{{sRGB},{linear}} \\B_{{sRGB},{linear}}\end{matrix}\rangle}} = {\langle\begin{matrix}R_{LED} \\G_{LED} \\B_{LED}\end{matrix}\rangle}}} & {{Equation}\mspace{14mu} 3}\end{matrix}$

In the above equation, the CCM is applied to the target RGB values(specified in sRGB color space without the gamma encoding) to producemodified target values. The modified target values correct for variancesbetween LEDs.

In some implementations, the calibrating device 106 sends commands tothe LED device to be calibrated (e.g., to enable LEDs). In someimplementations, the calibrating device 106 sends commands to the imagesensor (e.g., to capture image data). In some implementations, thecalibrating device 106 analyzes the image data received from the imagesensor. In some implementations, the calibrating device 106 detects adot of light for each LED to be calibrated and calculates the RGB valuesof each dot of light. In some implementations, the calibrating device106 calculates an LED brightness adjustment for each LED. In someimplementations, the calibrating device 106 calculates a colorcorrection matrix for each LED. In some implementations, the calibratingdevice 106 sends the LED brightness adjustments and color correctionmatrices to the LED device.

In some implementations, prior to calibrating the LEDs, the calibratingdevice 106 obtains one or more pictures from the image sensor to verifyor adjust the exposure level of the image sensor.

In some implementations, prior to obtaining the color matrices, thecalibrating device 106 obtains one or more pictures from the imagesensor and calibrates the LED(s) brightness based on the one or morepictures.

In some implementations, after sending the CCM(s) to the LED device, thecalibrating device 106 obtains one or more pictures from the imagesensor to verify that the corrected LED outputs (e.g., the outputs afterapplying the CCM(s)) are accurate (e.g., within a threshold distance ofthe desired values, such as 2%, 5%, or 10%).

It should be understood that the particular order in which theoperations in FIG. 1D have been described is merely an example and isnot intended to indicate that the described order is the only order inwhich the operations could be performed. For example, in someimplementations, operations 122 and 124 are performed after operation132. One of ordinary skill in the art would recognize various ways toreorder the operations described herein. Additionally, it should benoted that details of other processes described herein with respect toother methods and/or processes described herein are also applicable inan analogous manner to the method 120 described above with respect toFIG. 1D.

FIGS. 2A and 2B are a front view 200 and a rear view 220 of avoice-activated electronic device 190 in accordance with someimplementations. The electronic device 190 is designed as warm andinviting, and fits naturally in many areas of a home. The electronicdevice 190 includes one or more microphones 202 and an array of fullcolor LEDs 204. In some implementations, the full color LEDs 204 arelocated under a top surface of the electronic device 190 and invisibleto the user when they are not lit. In some implementations, the array offull color LEDs 204 is physically arranged in a ring. Further, the rearside of the electronic device 190 optionally includes a power supplyconnector 208 configured to couple to a power supply. In someimplementations, the full color LED comprises an LED with RGBcomponents, capable of displaying a plurality of colors.

In some implementations, the electronic device 190 presents a clean lookhaving no visible button, and the interaction with the electronic device190 is based on voice and touch gestures. Alternatively, in someimplementations, the electronic device 190 includes a limited number ofphysical buttons (e.g., a button 206 on its rear side), and theinteraction with the electronic device 190 is further based oninteraction with the physical buttons. In some implementations, one ormore speakers are disposed in the electronic device 190.

In some implementations, the electronic device 190 includes an array offull color LEDs 204, one or more microphones 202, a speaker 222,Dual-band WiFi 802.11ac radio(s), a Bluetooth LE radio, an ambient lightsensor, a USB port, a processor and memory storing at least one programfor execution by the processor.

Further, in some implementations, the electronic device 190 furtherincludes a touch sense array 224 configured to detect touch events onthe top surface of the electronic device 190. The touch sense array 224is disposed and concealed under the top surface of the electronic device190. In some implementations, the touch sense array 224 arranged on atop surface of a circuit board including an array of via holes, and thefull color LEDs are disposed within the via holes of the circuit board.When the circuit board is positioned immediately under the top surfaceof the electronic device 190, both the full color LEDs 204 and the touchsense array 224 are disposed immediately under the top surface of theelectronic device 190 as well.

In some implementations, given the simplicity and low cost of theelectronic device 190, the electronic device 190 includes an array offull color light emitting diodes (LEDs) rather than a full displayscreen. An LED design language is adopted to configure illumination ofthe array of full color LEDs and enable different visual patternsindicating different processing states of the electronic device 190. TheLED design language includes a grammar of colors, patterns, and specificmotion applied to a fixed set of full color LEDs. The elements in thelanguage are combined to visually indicate specific device states duringthe use of the electronic device 190. In some implementations,illumination of the full color LEDs delineates the passive listening andactive listening states of the electronic device 190 among otherimportant states. In some implementations, the array of full color LEDsis used in a speaker that is made by a third party original equipmentmanufacturer (OEM) based on specific technology (e.g., GoogleAssistant).

When the array of full color LEDs is used in a speaker that is made by athird party OEM based on specific technology, the full color LEDs andthe LED design language are configured to fit a corresponding physicaluser interface of the OEM speaker. In this situation, device states ofthe OEM speaker remain the same, while specific visual patternsrepresenting the device states are varied. For example, the colors ofthe full color LEDs are different but are displayed with similaranimation effects.

In a voice-activated electronic device 190, passive listening occurswhen the electronic device 190 processes audio inputs collected from itssurrounding environment but does not store the audio inputs or transmitthe audio inputs to any remote server. In contrast, active listeningoccurs when the electronic device 190 stores the audio inputs collectedfrom its surrounding environment and/or shares the audio inputs with aremote server. In accordance with some implementations of thisapplication, the electronic device 190 only passively listens to theaudio inputs in its surrounding environment without breaching privacy ofusers of the electronic device 190.

FIGS. 2C is a top view of a voice-activated electronic device 190 inaccordance with some implementations, and FIG. 2D shows six visualpatterns displayed by an array of full color LEDs for indicating voiceprocessing states in accordance with some implementations. In someimplementations, the electronic device 190 does not include any displayscreen, and the full color LEDs provide a simple and low cost visualuser interface compared with a full display screen. The full color LEDscould be hidden under a top surface of the electronic device andinvisible to the user when they are not lit. Referring to FIGS. 2C and2D, in some implementations, the array of full color LEDs are physicallyarranged in a ring.

Specifically, a method is implemented at the electronic device 190 forvisually indicating a processing state. The electronic device 190collects via the one more microphones audio inputs from an environmentin proximity to the electronic device, and processes the audio inputs.The processing includes one or more of identifying and responding tovoice inputs from a user in the environment. The electronic device 190determines a state of the processing from among a plurality ofpredefined voice processing states. For each of the full color LEDs, theelectronic device 190 identifies a respective predetermined LEDillumination specification associated with the determined voiceprocessing state. The illumination specification includes one or more ofan LED illumination duration, pulse rate, duty cycle, color sequence andbrightness. In some implementations, the electronic device 190determines that the voice processing state is associated with one of aplurality of users, and identifies the predetermined LED illuminationspecifications of the full color LEDs by customizing at least one of thepredetermined LED illumination specifications (e.g., the color sequence)of the full color LEDs according to an identity of the one of theplurality of users.

Further, in some implementations, in accordance with the determinedvoice processing state, the colors of the full color LEDs include apredetermined set of colors. For example, referring to FIGS. 2D(2),2D(2) and 2D(7)-(10), the predetermined set of colors include Googlebrand colors including blue, green, yellow and red, and the array offull color LEDs is divided into four quadrants each associated with oneof the Google brand colors.

In accordance with the identified LED illumination specifications of thefull color LEDs, the electronic device 190 synchronizes illumination ofthe array of full color LEDs to provide a visual pattern indicating thedetermined voice processing state. In some implementations, the visualpattern indicating the voice processing state includes a plurality ofdiscrete LED illumination pixels. In some implementations, the visualpattern includes a start segment, a loop segment and a terminationsegment. The loop segment lasts for a length of time associated with theLED illumination durations of the full color LEDs and configured tomatch a length of the voice processing state.

In some implementations, the electronic device 190 has more than twentydifferent device states (including the plurality of predefined voiceprocessing states) that are represented by the LED Design Language.Optionally, the plurality of predefined voice processing states includesone or more of a hot word detection state, a listening state, a thinkingstate and a responding state.

Accordingly, in some implementations, in accordance with a determinationthat the determined voice processing state is a hot word detection statethat occurs when one or more predefined hot words are detected, thearray of full color LEDs is divided into a plurality of diode groupsthat are alternately arranged and configured to be lit sequentially, anddiodes in each of the plurality of diode groups are lit with differentcolors. Further, in some implementations, in accordance with adetermination that the determined voice processing state is a listeningstate that occurs when the electronic device is actively receiving thevoice inputs from the environment and providing received voice inputs toa remote server, all full color LEDs are lit up with a single color, andeach full color LED illuminates with different and varying brightness.

In some implementations, the visual pattern is configured to beconsistent with human reactions (e.g., breathing, flickering, blinking,and swiping) associated with the voice processing state. For example,one of the most impactful places to use the Google brand colors, theattentive wake-up spin followed by the gentle breathing animationsignals patient, and eager, yet respectful listening. The colorsthemselves conjure a sense of brand and embodiment of the Google voiceassistant. These elements contrast with the dead front of the device toshow distinct “not recording” and “recording” states.

In some implementations, in accordance with a determination that thevoice processing state is a thinking state that occurs when theelectronic device is processing the voice inputs received from the user,an increasing number of RGB diodes are lit up during a firstillumination cycle of the LED illumination duration, and a decreasingnumber of RGB diodes are lit up during a second illumination cyclefollowing the first illumination cycle. Such a visual pattern isconsistent with a human reaction that a person is thinking. Optionally,the microphones are turned off in the thinking mode.

Referring to FIG. 2D(3), 2D(5) and 2D(6), motion most similar toprogress bars and other types of digital waiting signals are used in thevisual pattern to indicate the thinking mode in some implementations. Inaccordance with some implementations, white is used with the chasinganimation, brand colors are intentionally not used here to providebetter distinction contrast and highlighting with respect to the othervoice processing states.

In some implementations, in accordance with a determination that thevoice processing state is in a responding state that occurs when theelectronic device broadcasts a voice message in response to the voiceinputs received from the user, a subset of the full color LEDs are litup with a single color of distinct and varying brightness, and variationof the brightness of each of the subset of the fully color LEDs isconsistent with a voice speed associated with the voice inputs from theuser. In some implementations, a set of colors (e.g., the Google brandcolors) are used in the visual pattern to visually signify closure tothe voice query (e.g., that the question has been answered).

FIG. 3 is a block diagram illustrating an example electronic device 190that is applied as a voice interface to collect user voice commands in asmart media environment 100 in accordance with some implementations. Theelectronic device 190, typically, includes one or more processing units(CPUs) 302, one or more network interfaces 304, memory 306, and one ormore communication buses 308 for interconnecting these components(sometimes called a chipset). The electronic device 190 includes one ormore input devices 310 that facilitate user input, such as the button206, the touch sense array and the one or more microphones 202 shown inFIGS. 2A-2C. The electronic device 190 also includes one or more outputdevices 312, including one or more speakers 222 and the array of fullcolor LEDs 204.

Memory 306 includes high-speed random access memory, such as DRAM, SRAM,DDR RAM, or other random access solid state memory devices; and,optionally, includes non-volatile memory, such as one or more magneticdisk storage devices, one or more optical disk storage devices, one ormore flash memory devices, or one or more other non-volatile solid statestorage devices. Memory 306, optionally, includes one or more storagedevices remotely located from one or more processing units 302. Memory306, or alternatively the non-volatile memory within memory 306,includes a non-transitory computer-readable storage medium. In someimplementations, memory 306, or the non-transitory computer-readablestorage medium of memory 306, stores the following programs, modules,and data structures, or a subset or superset thereof:

-   -   Operating system 316 including procedures for handling various        basic system services and for performing hardware dependent        tasks;    -   Network communication module 318 for connecting the electronic        device 190 to other devices (e.g., the server system(s), the        client device(s), and the like) via one or more network        interfaces 304 (wired or wireless) and one or more networks,        such as the Internet, other wide area networks, local area        networks, metropolitan area networks, and so on;    -   Input/output control module for receiving inputs via one or more        input devices 310 enabling presentation of information at the        electronic device 190 via one or more output devices 312,        including:        -   Voice processing module 322 for processing audio inputs or            voice messages collected in an environment surrounding the            electronic device 190, or preparing the collected audio            inputs or voice messages for processing at a voice            assistance server or a cloud cast service server;        -   LED control module 324 for governing operation of the LEDs            204, including generating visual patterns on the LEDs 204            according to device states of the electronic device 190 and            applying correction parameters to the LEDs 204 (e.g.,            adjusting color and/or brightness); and        -   Touch sense module 326 for sensing touch events on a top            surface of the electronic device 190;    -   Voice activated device data 330 storing at least data associated        with the electronic device 190, including:        -   Voice device settings 332 for storing information associated            with the electronic device 190 itself, including common            device settings (e.g., service tier, device model, storage            capacity, processing capabilities, communication            capabilities, etc.), information of a user account in a user            domain, and display specifications associated with one or            more visual patterns displayed by the full color LEDs; and        -   Voice control data 334 for storing audio signals, voice            messages, response messages and other data related to voice            interface functions of the electronic device 190; and    -   LED configuration data 336 for configuring the various LED        parameters, such as brightness and color values, including:        -   Color correction matrices 337 to enable color and/or            brightness of the full color LEDs to be corrected during            illumination operations, such as operations related to            conveying state of a voice processing operation or of a            particular electronic device.

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices, and corresponds to a set ofinstructions for performing a function described above. The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures, modules or datastructures, and thus various subsets of these modules may be combined orotherwise re-arranged in various implementations. In someimplementations, memory 306, optionally, stores a subset of the modulesand data structures identified above. Furthermore, memory 306,optionally, stores additional modules and data structures not describedabove.

FIG. 4 is a flow diagram illustrating a method 400 of visuallyindicating a voice processing state in accordance with someimplementations. The method 400 is implemented at an electronic device190 with an array of full color LEDs, one or more microphones, aspeaker, a processor and memory storing at least one program forexecution by the processor. The electronic device 190 collects (402) viathe one more microphones 402 audio inputs from an environment inproximity to the electronic device 190, and processes (404) the audioinputs. The processing is implemented at voice processing module 522,and includes one or more of identifying and responding to voice inputsfrom a user in the environment. The electronic device 190 thendetermines (406) a state of the processing from among a plurality ofpredefined voice processing states. For each of the full color LEDs, theelectronic device 190 identifies (408) a respective predetermined LEDillumination specification associated with the determined voiceprocessing state, and the respective illumination specification includes(410) one or more of an LED illumination duration, pulse rate, dutycycle, color sequence and brightness. In accordance with the identifiedLED illumination specifications of the full color LEDs, the electronicdevice 190 (specifically, LED control module 524) synchronizesillumination of the array of full color LEDs to provide a visual patternindicating the determined voice processing state. More details on themethod 400 have been explained above with reference to FIGS. 4A-4G and5.

Method 400 is, optionally, governed by instructions that are stored in anon-transitory computer-readable storage medium and that are executed byone or more processors of a voice-activated electronic device 190. Eachof the operations shown in FIG. 4 may correspond to instructions storedin the computer memory or computer-readable storage medium (e.g., memory306 of the electronic device 190 in FIG. 3). The computer-readablestorage medium may include a magnetic or optical disk storage device,solid state storage devices such as Flash memory, or other non-volatilememory device or devices. The computer-readable instructions stored onthe computer-readable storage medium may include one or more of: sourcecode, assembly language code, object code, or other instruction formatthat is interpreted by one or more processors. Some operations in themethod 400 may be combined and/or the order of some operations may bechanged.

FIG. 5 is a block diagram illustrating an example calibrating device 106in accordance with some implementations. The calibrating device 106,typically, includes one or more processing units (CPUs) 502, one or morenetwork interfaces 504, memory 506, and one or more communication buses508 for interconnecting these components (sometimes called a chipset).The calibrating device 106 typically also includes one or more inputdevices 510 that facilitate user input and one or more output devices512.

Memory 506 includes high-speed random access memory, such as DRAM, SRAM,DDR RAM, or other random access solid state memory devices; and,optionally, includes non-volatile memory, such as one or more magneticdisk storage devices, one or more optical disk storage devices, one ormore flash memory devices, or one or more other non-volatile solid statestorage devices. Memory 506, optionally, includes one or more storagedevices remotely located from one or more processing units 502. Memory506, or alternatively the non-volatile memory within memory 506,includes a non-transitory computer-readable storage medium. In someimplementations, memory 506, or the non-transitory computer-readablestorage medium of memory 506, stores the following programs, modules,and data structures, or a subset or superset thereof:

-   -   Operating system 516 including procedures for handling various        basic system services and for performing hardware dependent        tasks;    -   Network communication module 518 for communicatively connecting        the calibrating device 106 to other devices (e.g., the server        system(s), the client device(s), and the like) via one or more        network interfaces 504 (wired or wireless) and one or more        networks, such as the Internet, other wide area networks, local        area networks, metropolitan area networks, and so on;    -   Input/output control module 520 for receiving inputs via the one        or more input devices 510 and enabling presentation of        information at the calibrating device 106 via the one or more        output devices 512;    -   LED calibration module 524 for analyzing image information and        generating LED calibration information, such as CCM(s); and    -   LED configuration data 526 for use with calibrating LEDs,        optionally including de-gamma data, desired LED color values,        desired LED brightness values, image sensor color space        information, and the like.

Each of the above identified elements may be stored in one or more ofthe previously mentioned memory devices, and corresponds to a set ofinstructions for performing a function described above. The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures, modules or datastructures, and thus various subsets of these modules may be combined orotherwise re-arranged in various implementations. In someimplementations, memory 506, optionally, stores a subset of the modulesand data structures identified above. Furthermore, memory 506,optionally, stores additional modules and data structures not describedabove.

Although various drawings illustrate a number of logical stages in aparticular order, stages that are not order dependent may be reorderedand other stages may be combined or broken out. While some reordering orother groupings are specifically mentioned, others will be obvious tothose of ordinary skill in the art, so the ordering and groupingspresented herein are not an exhaustive list of alternatives. Moreover,it should be recognized that the stages can be implemented in hardware,firmware, software or any combination thereof.

The terminology used in the description of the various describedimplementations herein is for the purpose of describing particularimplementations only and is not intended to be limiting. As used in thedescription of the various described implementations and the appendedclaims, the singular forms “a”, “an” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will also be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items. It will be furtherunderstood that the terms “includes,” “including,” “comprises,” and/or“comprising,” when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof.

As used herein, the term “if” is, optionally, construed to mean “when”or “upon” or “in response to determining” or “in response to detecting”or “in accordance with a determination that,” depending on the context.Similarly, the phrase “if it is determined” or “if [a stated conditionor event] is detected” is, optionally, construed to mean “upondetermining” or “in response to determining” or “upon detecting [thestated condition or event]” or “in response to detecting [the statedcondition or event]” or “in accordance with a determination that [astated condition or event] is detected,” depending on the context.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific implementations. However, theillustrative discussions above are not intended to be exhaustive or tolimit the scope of the claims to the precise forms disclosed. Manymodifications and variations are possible in view of the aboveteachings. The implementations were chosen in order to best explain theprinciples underlying the claims and their practical applications, tothereby enable others skilled in the art to best use the implementationswith various modifications as are suited to the particular usescontemplated.

What is claimed is:
 1. An electronic device, comprising: a plurality oflight emitting diodes (LEDs); one or more processors; and memory storinga plurality of color correction matrices, each color correction matrixof the plurality of color correction matrices: corresponding to an LEDof the plurality of LEDs, and generated based on a desired color valuefor the corresponding LED, wherein the electronic device is configuredto relay status of the electronic device via the plurality of LEDsoperating in conjunction with the plurality of color correctionmatrices.
 2. The electronic device of claim 1, wherein each LED of theplurality of LEDs is a full color LED.
 3. The electronic device of claim1, wherein each LED of the plurality of LEDs has red, green, and blue(RGB) values that are modified by application of the corresponding colorcorrection matrix.
 4. The electronic device of claim 1, wherein each LEDof the plurality of LEDs is uncalibrated.
 5. The electronic device ofclaim 1, wherein each LED of the plurality of LEDs is configured tooutput light having one or more color components of a plurality of colorcomponents; and wherein the plurality of color correction matricesincludes a color correction matrix corresponding to each color componentof the one or more color components.
 6. The electronic device of claim1, wherein the electronic device is a voice-activated device.
 7. Theelectronic device of claim 1, wherein the memory stores one or moreapplications, the one or more applications including instructions tooperate the plurality of LEDs in conjunction with the plurality of colorcorrection matrices.
 8. The electronic device of claim 7, wherein theinstructions to operate the plurality of LEDs include instructionsregarding one or more of: illumination duration, pulse rate, duty cycle,color sequence, and brightness.
 9. The electronic device of claim 7,wherein the instructions to operate the plurality of LEDs includesettings for one or more operating parameters of each LED of theplurality of LEDs.
 10. The electronic device of claim 9, wherein the oneor more operating parameters include a brightness parameter.
 11. Theelectronic device of claim 1, wherein the memory stores instructions foroperating the plurality of LEDs in conjunction with the plurality ofcolor correction matrices in accordance with a color design language.12. The electronic device of claim 1, wherein each color correctionmatrix of the plurality of color correction matrices is scaled toprevent overrange of color values for the corresponding LED.